Sirt2 is a target for the treatment of neurological, metabolic, and age-related diseases including cancer. Here we report a series of Sirt2 inhibitors based on the 1,2,4-oxadiazole scaffold. These compounds are potent Sirt2 inhibitors active at single-digit μM level by using the Sirt2 substrate α-tubulin-acetylLys40 peptide and inactive up to 100 μM against Sirt1, -3, and -5 (deacetylase and desuccinylase activities). Their mechanism of inhibition is uncompetitive toward both the peptide substrate and NAD+, and the crystal structure of a 1,2,4-oxadiazole analog in complex with Sirt2 and ADP-ribose reveals its orientation in a still unexplored subcavity useful for further inhibitor development. Tested in leukemia cell lines, 35 and 39 induced apoptosis and/or showed antiproliferative effects at 10 or 25 μM after 48 h. Western blot analyses confirmed the involvement of Sirt2 inhibition for their effects in NB4 and in U937 cells. Our results provide novel Sirt2 inhibitors with a compact scaffold and structural insights for further inhibitor improvement.